Approximate Dynamic Programming Recurrence Relations for a Hybrid Optimal Control Problem

نویسندگان

  • W. Lu
  • S. Ferrari
  • R. Fierro
  • T. A. Wettergren
چکیده

This paper presents a hybrid approximate dynamic programming (ADP) method for a hybrid dynamic system (HDS) optimal control problem, that occurs in many complex unmanned systems which are implemented via a hybrid architecture, regarding robot modes or the complex environment. The HDS considered in this paper is characterized by a well-known three-layer hybrid framework, which includes a discrete event controller layer, a discrete-continuous interface layer, and a continuous state layer. The hybrid optimal control problem (HOCP) is to find the optimal discrete event decisions and the optimal continuous controls subject to a deterministic minimization of a scalar function regarding the system state and control over time. Due to the uncertainty of environment and complexity of the HOCP, the cost-to-go cannot be evaluated before the HDS explores the entire system state space; as a result, the optimal control, neither continuous nor discrete, is not available ahead of time. Therefore, ADP is adopted to learn the optimal control while the HDS is exploring the environment, because of the online advantage of ADP method. Furthermore, ADP can break the curses of dimensionality which other optimizing methods, such as dynamic programming (DP) and Markov decision process (MDP), are facing due to the high dimensions of HOCP.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Stochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry

We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...

متن کامل

OPTIMIZATION OF A PRODUCTION LOT SIZING PROBLEM WITH QUANTITY DISCOUNT

Dynamic lot sizing problem is one of the significant problem in industrial units and it has been considered by  many researchers. Considering the quantity discount in  purchasing cost is one of the important and practical assumptions in the field of inventory control models and it has been less focused in terms of stochastic version of dynamic lot sizing problem. In  this paper, stochastic dyn...

متن کامل

A Hybrid Dynamic Programming for Inventory Routing Problem in Collaborative Reverse Supply Chains

Inventory routing problems arise as simultaneous decisions in inventory and routing optimization. In the present study, vendor managed inventory is proposed as a collaborative model for reverse supply chains and the optimization problem is modeled in terms of an inventory routing problem. The studied reverse supply chains include several return generators and recovery centers and one collection...

متن کامل

A Novel Hybrid Modified Binary Particle Swarm Optimization Algorithm for the Uncertain p-Median Location Problem

Here, we investigate the classical p-median location problem on a network in which the vertex weights and the distances between vertices are uncertain. We propose a programming model for the uncertain p-median location problem with tail value at risk objective. Then, we show that it is NP-hard. Therefore, a novel hybrid modified binary particle swarm optimization algorithm is presented to obtai...

متن کامل

Hybrid model predictive control of a nonlinear three-tank system based on the proposed compact form of piecewise affine model

In this paper, a predictive control based on the proposed hybrid model is designed to control the fluid height in a three-tank system with nonlinear dynamics whose operating mode depends on the instantaneous amount of system states. The use of nonlinear hybrid model in predictive control leads to a problem of mixed integer nonlinear programming (MINLP) which is very complex and time consuming t...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2012